Sigma Delta Quantized Networks

نویسندگان

  • Peter O'Connor
  • Max Welling
چکیده

Deep neural networks can be obscenely wasteful. When processing video, a convolutional network expends a fixed amount of computation for each frame with no regard to the similarity between neighbouring frames. As a result, it ends up repeatedly doing very similar computations. To put an end to such waste, we introduce Sigma-Delta networks. With each new input, each layer in this network sends a discretized form of its change in activation to the next layer. Thus the amount of computation that the network does scales with the amount of change in the input and layer activations, rather than the size of the network. We introduce an optimization method for converting any pre-trained deep network into an optimally efficient Sigma-Delta network, and show that our algorithm, if run on the appropriate hardware, could cut at least an order of magnitude from the computational cost of processing video data.

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Sigma-Delta quantization of sub-Gaussian frame expansions and its application to compressed sensing

Suppose that the collection {ei}i=1 forms a frame for R, where each entry of the vector ei is a sub-Gaussian random variable. We consider expansions in such a frame, which are then quantized using a Sigma-Delta scheme. We show that an arbitrary signal in R can be recovered from its quantized frame coefficients up to an error which decays root-exponentially in the oversampling rate m/k. Here the...

متن کامل

Approximating a Bandlimited Function Using Very Coarsely Quantized Data: Improved Error Estimates in Sigma-delta Modulation

This paper concerns fine analytical error estimates in analog-to-digital conversion of bandlimited functions using the method of sigma-delta modulation (also called Σ∆ quantization). This method has found widespread usage in practice due to several advantages in its implementation compared to conventional methods (see [1, 11]). A recent mathematical treatment of this problem appears in [3]. Bel...

متن کامل

Generalized sigma-derivation on Banach algebras

Let $mathcal{A}$ be a Banach algebra and $mathcal{M}$ be a Banach $mathcal{A}$-bimodule. We say that a linear mapping $delta:mathcal{A} rightarrow mathcal{M}$ is a generalized $sigma$-derivation whenever there exists a $sigma$-derivation $d:mathcal{A} rightarrow mathcal{M}$ such that $delta(ab) = delta(a)sigma(b) + sigma(a)d(b)$, for all $a,b in mathcal{A}$. Giving some facts concerning general...

متن کامل

Time-Mode Signal Quantization for Use in Sigma-Delta Modulators

The rapid scaling in modern CMOS technology has motivated the researchers to design new analog-to-digital converter (ADC) architectures that can properly work in lower supply voltage. An exchanging the data quantization procedure from the amplitude to the time domain, can be a promising alternative well adapt with the technology scaling. This paper is going to review the recent development in t...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

عنوان ژورنال:
  • CoRR

دوره abs/1611.02024  شماره 

صفحات  -

تاریخ انتشار 2016